Image processing apparatus, method and program for gradation conversion

ABSTRACT

An image processing apparatus includes a first calculating unit configured to add a pixel value and an output of a filter unit, a first quantizing unit configured to quantize an output of the first calculating unit and output a quantized value serving as ΔΣ modulation data, a second calculating unit configured to calculate a difference between the output of the first calculating unit and the quantized value, thereby obtaining the quantization error, a second quantizing unit configured to quantize a portion of the quantization error and output compensating data, a third calculating unit configured to add the ΔΣ modulation data and the compensating data, thereby generating time-integration-effect-using error diffusion data, a fourth calculating unit configured to calculate a difference between the quantization error and the compensating data, the difference serving as a ΔΣ modulation error, and the filter unit configured to perform filtering in space directions.

CROSS-REFERENCE TO RELATED APPLICATION

The present application claims priority from Japanese Patent ApplicationNo. JP 2008-272169 filed in the Japanese Patent Office on Oct. 22, 2008,the entire content of which is incorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image processing apparatus, an imageprocessing method, and a program. Particularly, the present inventionrelates to an image processing apparatus, an image processing method,and a program that enable an improvement in perceived image qualityafter gradation conversion.

2. Description of the Related Art

For example, in a case where an image of a large number of bits, such asan image in which each of RGB (Red, Green, and Blue) values is 8 bits,is to be displayed on a display of a small number of bits, such as anLCD (Liquid Crystal Display) capable of displaying an image in whicheach of RGB values is 6 bits, it is necessary to perform gradationconversion for converting the gradation level of the image.

An example of a method for performing the gradation conversion is an FRC(Frame Rate Control) process.

In the FRC process, the frame rate of images to be displayed on adisplay is adjusted to match the display rate of the display, thedisplay rate being four times higher than the frame rate, for example,and then the images are displayed on the display.

That is, for example, assume that 8-bit images are to be displayed on a6-bit LCD. When the focus is put on a pixel in a frame of the 8-bitimages, the frame is called a target frame and the pixel is called atarget pixel.

Also, assume that the frame rate (or the field rate) of the 8-bit imagesis 60 Hz and that the display rate of the 6-bit LCD is four times theframe rate of the 8-bit images, that is, 240 Hz.

In the FRC process, the frame rate of the images is controlled to befour times so that the frame rate matches the display rate of thedisplay, and then images having a frame rate that has been controlledare displayed.

That is, four (=240 Hz/60 Hz) 1-bit values that correspond to lower two(=8-6) bits among 8 bits in total of a pixel value of the target pixelare represented by (b₀, b₁, b₂, and b₃).

A 6-bit value obtained by truncating the lower 2 bits of the 8-bit pixelvalue (the value obtained by simply quantizing the 8-bit pixel valueinto a 6-bit pixel value) is represented by X.

In the FRC process, basically, an 8-bit pixel value is converted intofour 6-bit pixel values (the pixel values at the position of the targetpixel in sequential four frames) X+b₀, X+b₁, X+b₂, and X+b₃.

Specifically, in a case where the 8-bit pixel value of the target pixelis 127 (=01111111B), B indicating that the preceding value is a binarynumber, a 6-bit value X obtained by truncating the lower 2 bits of the8-bit pixel value is 31 (=011111B).

Also, as the four 1-bit values (b₀, b₁, b₂, and b₃) that correspond tothe lower two bits 11B (=3) among 8 bits in total of the pixel value 127(=01111111B) of the target pixel, (0B, 1B, 1B, and 1B) are used, forexample.

Therefore, the 8-bit pixel value 127 (=01111111B) of the target pixel isconverted into four 6-bit pixel values X+b₀=31 (=011111B), X+b₁=32(=100000B), X+b₂=32 (=100000B), and X+b₃=32 (=100000B).

In the FRC process, a target frame is converted into four frames so thata frame rate matches the display rate of an LCD. Now, assume that thefour frames are called first, second, third, and fourth frames indisplay time series. In this case, the pixel values of pixels at theposition of the target pixel in the first to fourth frames correspond tothe above-described four 6-bit pixel values 31, 32, 32, and 32 in theFRC process.

In the FRC process, the first to fourth frames are displayed on the LCDat a display rate four times the original frame rate. In this case, atthe position of the target pixel, the 6-bit pixel values 31, 32, 32, and32 are integrated (added) in a time direction in human vision, so thatthe pixel value looks like 127.

As described above, in the FRC process, 127 as an 8-bit pixel value isexpressed by 6 bits in a pseudo manner with use of a time integrationeffect in which integration in a time direction is performed in humanvision.

In the FRC process, a process of converting an 8-bit pixel value intofour 6-bit pixel values is performed by using an LUT (Look Up Table)storing the 8-bit pixel value and the four 6-bit pixel values that aremutually associated.

Another example of the method for performing the gradation conversion isan error diffusion method (e.g., see “Yoku wakaru dijitaru gazou shori”by Hitoshi KIYA, Sixth edition, CQ Publishing, Co. Ltd., January 2000,pp. 196-213).

In gradation conversion based on the error diffusion method, noiseshaping to a high range of spatial frequencies is performed on noise,which is a quantization error of a pixel value of a pixel that isspatially approximate to a target pixel, and the noise on which noiseshaping has been performed is added to the pixel value of the targetpixel, whereby error diffusion is performed (error diffusion of adding aquantization error of a pixel value of a target pixel to a pixel valueof a pixel that is spatially approximate to the target pixel). Then, thepixel value to which the noise has been added is quantized into adesired number of bits.

The gradation conversion based on the error diffusion method istwo-dimensional ΔΣ modulation in space directions, in which a pixelvalue is quantized after noise (quantization error) has been addedthereto, as described above. Therefore, in a quantized(gradation-converted) image, it looks like PWM (Pulse Width Modulation)has been performed on pixel values that become constant only bytruncating lower bits. As a result, the gradation of agradation-converted image looks like it smoothly changes due to a spaceintegration effect in which integration in space directions is performedin human vision. That is, a gradation level equivalent to that of anoriginal image (e.g., 256 (2⁸)-gradation when the original image is an8-bit image as described above) can be expressed in a pseudo manner.

Also, in the error diffusion method, noise (quantization error) afternoise shaping is added to a pixel value in consideration that thesensitivity of human vision is low in a high range of spatialfrequencies. Accordingly, the level of noise noticeable in agradation-converted image can be decreased.

SUMMARY OF THE INVENTION

In the error diffusion method, noise after noise shaping is added to apixel value and thus the level of noise noticeable in agradation-converted image can be decreased, as described above. However,when the amount of noise added to a pixel value is large, the noiseadded to the pixel value, that is, the noise diffused in spacedirections, can be noticeable in a gradation-converted image.

As a method for preventing noise from being noticeable in agradation-converted image, a method for reducing the amount of noisethat is diffused in space directions can be accepted. In this method,however, an effect of error diffusion (an effect of the error diffusionmethod) is insufficient, and a perceived gradation level decreases.

Accordingly, it is desirable to improve a perceived image quality bypreventing noise from being noticeable in a gradation-converted imagewithout causing a decrease in perceived gradation level of thegradation-converted image.

According to an embodiment of the present invention, there is providedan image processing apparatus including first calculating means foradding a pixel value of an image and an output of filter means forperforming filtering in space directions on a quantization error of aquantized value obtained by quantizing the pixel value of the image,first quantizing means for quantizing an output of the first calculatingmeans and outputting a quantized value including the quantization error,the quantized value serving as ΔΣ modulation data, which is a result ofΔΣ modulation performed on the pixel value, second calculating means forcalculating a difference between the output of the first calculatingmeans and the quantized value of the output of the first calculatingmeans, thereby obtaining the quantization error, second quantizing meansfor quantizing a portion of the quantization error and outputting aquantized value obtained through the quantization, the quantized valueserving as compensating data for compensating for error diffusion inspace directions, third calculating means for adding the ΔΣ modulationdata and the compensating data, thereby generatingtime-integration-effect-using error diffusion data that generates aneffect of an error diffusion method using a visual integration effect ina time direction, fourth calculating means for calculating a differencebetween the quantization error and the compensating data, the differenceserving as a ΔΣ modulation error, which is a quantization error used forthe ΔΣ modulation, and the filter means for performing filtering inspace directions on the ΔΣ modulation error. Also, there is provided aprogram causing a computer to function as the image processingapparatus.

According to an embodiment of the present invention, there is providedan image processing method including the steps of adding a pixel valueof an image and an output of filter means, the adding being performed byfirst calculating means, quantizing an output of the first calculatingmeans and outputting a quantized value including a quantization error,the quantized value serving as ΔΣ modulation data, the quantizing andthe outputting being performed by first quantizing means, calculating adifference between the output of the first calculating means and thequantized value of the output of the first calculating means, therebyobtaining the quantization error, the calculating being performed bysecond calculating means, quantizing a portion of the quantization errorand outputting compensating data, the quantizing and the outputtingbeing performed by second quantizing means, adding the ΔΣ modulationdata and the compensating data, thereby generatingtime-integration-effect-using error diffusion data, the adding beingperformed by third calculating means, calculating a difference betweenthe quantization error and the compensating data, the difference servingas a ΔΣ modulation error, the calculating being performed by fourthcalculating means, and performing filtering in space directions on theΔΣ modulation error, the performing being performed by the filter means.

In the foregoing image processing apparatus, image processing method,and program, the first calculating means adds a pixel value of an imageand an output of the filter means, and the first quantizing meansquantizes an output of the first calculating means and outputs ΔΣmodulation data. Furthermore, the second calculating means calculates adifference between the output of the first calculating means and thequantized value of the output of the first calculating means, therebyobtaining a quantization error. The second quantizing means quantizes aportion of the quantization error and outputs compensating data. Thethird calculating means adds the ΔΣ modulation data and the compensatingdata, thereby generating time-integration-effect-using error diffusiondata. The fourth calculating means calculates a difference between thequantization error and the compensating data, the difference serving asa ΔΣ modulation error. The filter means performs filtering in spacedirections on the ΔΣ modulation error.

The image processing apparatus may be an independent apparatus or may bean internal block constituting an apparatus.

The program can be provided by being transmitted via a transmissionmedium or by being recorded on a recording medium.

According to the above-described embodiments of the present invention, aperceived image quality after gradation conversion can be improved.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating an exemplary configuration of animage processing apparatus according to an embodiment of the presentinvention;

FIG. 2 is a block diagram illustrating an exemplary configuration of agradation converting unit of the image processing apparatus;

FIG. 3 is a block diagram illustrating an exemplary configuration of adata processing unit of the gradation converting unit;

FIGS. 4A to 4C illustrate data handled in the gradation converting unit;

FIG. 5 is a flowchart illustrating data processing performed by the dataprocessing unit;

FIG. 6 illustrates an amplitude characteristic of noise shaping using aJarvis filter and an amplitude characteristic of noise shaping using aFloyd filter;

FIG. 7 illustrates an amplitude characteristic of noise shaping usingthe Jarvis filter and an amplitude characteristic of noise shaping usingthe Floyd filter;

FIG. 8 illustrates an amplitude characteristic of noise shaping using anSBM filter;

FIG. 9 illustrates a quantization error used for filtering;

FIGS. 10A and 10B illustrate a first example of filter coefficients andan amplitude characteristic of noise shaping using the SBM filter;

FIGS. 11A and 11B illustrate a second example of filter coefficients andan amplitude characteristic of noise shaping using the SBM filter;

FIGS. 12A and 12B illustrate a third example of filter coefficients andan amplitude characteristic of noise shaping using the SBM filter; and

FIG. 13 is a block diagram illustrating an exemplary configuration of acomputer according to an embodiment of the present invention.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Exemplary configuration of an image processing apparatus according to anembodiment of the present invention

FIG. 1 is a block diagram illustrating an exemplary configuration of animage processing apparatus according to an embodiment of the presentinvention.

The image processing apparatus in FIG. 1 includes a gradation convertingunit 11 and a display 12, and is applied to a television receiver(hereinafter referred to as TV) or the like.

The gradation converting unit 11 is supplied with, as target image data,image data in which each of RGB components is 8 bits. The gradationconverting unit 11 performs gradation conversion of converting the 8-bittarget image data supplied thereto into 6-bit image data (image data inwhich each of RGB components is 6 bits) that can be displayed on thedisplay 12, and supplies the gradation-converted 6-bit image data to thedisplay 12.

In this embodiment, the frame rate (or the field rate) of the 8-bittarget image data is 60 Hz, for example, and the display rate of thedisplay 12 that displays 6-bit image data is 240 Hz, four times theframe rate of the 8-bit target image data.

Also, the gradation converting unit 11 performs frame rate conversionduring gradation conversion of the 8-bit target image data into 6-bitimage data (in this case, the frame rate is converted from 60 Hz, whichis the frame rate of the target image data, into 240 Hz, which is thedisplay rate of the display 12).

The display 12 is a 6-bit LCD capable of displaying 6-bit image data ata display rate of 240 Hz, and displays (images corresponding to) 6-bitimage data supplied from the gradation converting unit 11.

In the gradation converting unit 11, gradation conversion of the 8-bittarget image data is performed independently for each of RGB components,for example.

Exemplary Configuration of the Gradation Converting Unit 11

FIG. 2 illustrates an exemplary configuration of the gradationconverting unit 11 in FIG. 1.

The gradation converting unit 11 includes a data processing unit 21 andan FRC unit 22.

The data processing unit 21 is supplied with 8-bit target image data.The data processing unit 21 performs predetermined data processing onthe 8-bit target image data, thereby generating image data serving astime-integration-effect-using error diffusion data for generating aneffect of the error diffusion method by using a time integration effect(visual integration effect in a time direction).

The time-integration-effect-using error diffusion data that is generatedfrom the 8-bit target image data by the data processing unit 21 is 8-bitimage data in which the number of bits (of a pixel value) is the same asthat of the target image data, and is image data in which the frame rateis also the same as that of the target image data (60 Hz).

The time-integration-effect-using error diffusion data, which is imagedata in which the number of bits is eight and the frame rate is 60 Hz,is supplied from the data processing unit 21 to the FRC unit 22.

The FRC unit 22 performs an FRC process to convert thetime-integration-effect-using error diffusion data supplied from thedata processing unit 21, which is image data in which the number of bitsis eight and the frame rate is 60 Hz, into image data in which thenumber of bits is six and the frame rate is 240 Hz. Then, the FRC unit22 supplies the image data as gradation-converted image data to thedisplay 12.

Each of the data processing unit 21 and the FRC unit 22 can be realizedby dedicated hardware or software.

However, since the FRC process has already been realized by hardware,existing hardware can be adopted for the FRC unit 22.

In a case where the FRC unit 22 is realized by software instead of byexisting software, it is necessary for the FRC unit 22 to be configuredto operate at a speed four times that of the data processing unit 21because the FRC unit handles image data having a frame rate of 240 Hz,four times the frame rate of image data handled by the data processingunit 21.

Exemplary Configuration of the Data Processing Unit 21

FIG. 3 illustrates an exemplary configuration of the data processingunit 21 in FIG. 2.

Referring to FIG. 3, the data processing unit 21 includes an calculatingunit 31, a quantizing unit 32, calculating units 33 and 34, a filter 35,a calculating unit 36, a quantizing unit 37, a dequantizing unit 38, anda calculating unit 39.

The calculating unit 31 is supplied with pixel values IN(x, y) of N (=8)bits of pixels in target image data in a raster scanning order.Furthermore, the calculating unit 31 is supplied with outputs of thefilter 35, which performs filtering in space directions on aquantization error of a quantized value obtained by quantizing a pixelvalue of the target image data.

The calculating unit 31 adds the pixel value IN(x, y) of the targetimage data and the output of the filter 35 and supplies (outputs) a sumvalue U(x, y) obtained thereby to the quantizing unit 32 and thecalculating unit 33.

Here, IN(x, y) represents a pixel value of the pixel (x, y) x-th fromthe left and y-th from the top. U(x, y) represents a sum value of thepixel value IN(x, y) and the output of the filter 35.

The quantizing unit 32 quantizes the sum value U(x, y), which is theoutput of the calculating unit 31, into the number of bits smaller thanthe number of bits N (=8) of the target image data, i.e., into thenumber of bits M (=6) of an image that can be displayed on the display12 (FIG. 1), and then outputs a quantized value of M (=6) bits obtainedthereby as ΔΣ modulation data ID(x, y), which is a result of ΔΣmodulation performed on the pixel value IN (x, y).

In the data processing unit 21 in FIG. 3, the calculating unit 31 andthe quantizing unit 32 described above and the calculating unit 33 andthe filter 35 described below constitute a ΔΣ modulator that performs ΔΣmodulation, and the output of the quantizing unit 32 is a result of ΔΣmodulation performed on the pixel value IN(x, y) supplied to thecalculating unit 31.

The ΔΣ modulation data ID(x, y) of M (=6) bits output from thequantizing unit 32 is supplied to the calculating unit 33 and thedequantizing unit 38.

The calculating unit 33 calculates a difference U(x, y)−ID(x, y) betweenthe sum value U(x, y), which is the output of the calculating unit 31,and the ΔΣ modulation data ID(x, y) of M (=6) bits, which is the outputof the quantizing unit 32 and which is a quantized value of the sumvalue U(x, y), thereby obtaining a quantization error Q(x, y) includedin the ΔΣ modulation data ID(x, y) as a quantized value, and outputs thequantization error Q(x, y).

The quantization error Q(x, y) output from the calculating unit 33 issupplied to the calculating units 34 and 36.

The calculating unit 34 is supplied with compensating data Qt(x, y),which is a quantized value of a portion of the quantization error Q(x,y), from the quantizing unit 37, in addition to the quantization errorQ(x, y) output from the calculating unit 33.

The calculating unit 34 calculates a difference Q(x, y)−Qt(x, y) betweenthe quantization error Q(x, y) supplied from the calculating unit 33 andthe compensating data Qt(x, y) supplied from the quantizing unit 37, thedifference being regarded as a ΔΣ modulation error Qs(x, y) serving as aquantization error used for ΔΣ modulation performed in the ΔΣ modulator,and supplies the ΔΣ modulation error Qs(x, y) to the filter 35.

The filter 35 is an FIR (Finite Impulse Response) filter for performingtwo-dimensional filtering in space directions (horizontal and verticaldirections), and performs filtering in space directions (hereinafterreferred to as space-direction filtering) on the ΔΣ modulation errorQs(x, y), which is a quantization error supplied from the calculatingunit 34. Furthermore, the filter 35 supplies (outputs) a filteringresult to the calculating unit 31.

Here, a transfer function of the filter 35 is represented by G. In thiscase, the ΔΣ modulation data ID(x, y) output from the quantizing unit 32is expressed by expression (1).ID(x,y)=IN(x,y)−(1−G)k′Q(x,y)  (1)

In expression (1), the quantization error Q (x, y) is modulated with−(1−G)k′. The modulation with −(1−G)k′ corresponds to noise shapingbased on ΔΣ modulation in space directions.

In FIG. 3, a value obtained by subtracting the compensating data Qt(x,y) obtained in the quantizing unit 37 from the quantization error Q(x,y) is used as the ΔΣ modulation error Qs(x, y).

Now, assume that a weight having a value in the range from 0 to 1 isrepresented by k′ and that the ΔΣ modulation error Qs(x, y) is expressedby an expression Qs(x, y)=k′×Q(x, y) using the quantization error Q(x,y). In this case, in the data processing unit 21 in FIG. 3, only aportion for the weight k′ (ΔΣ modulation error Qs(x, y)) of thequantization error Q(x, y) as noise is used for ΔΣ modulation (noiseshaping based on ΔΣ modulation is performed).

Therefore, the ΔΣ modulation data ID(x, y) obtained through such ΔΣmodulation is data in which error diffusion in space directions isperformed on only a portion for the weight k′ of the quantization errorQ(x, y) in the target image data.

The calculating unit 36 extracts a portion of the quantization errorQ(x, y) supplied from the calculating unit 33 and supplies the extractedportion to the quantizing unit 37. Specifically, the calculating unit 36multiplies the quantization error Q(x, y) supplied from the calculatingunit 33 by a weight k having a value in the range from 0 to 1, therebyextracting a portion k×Q(x, y) of the quantization error Q(x, y), andsupplies the portion k×Q(x, y) to the quantizing unit 37.

The weight k used in the calculating unit 36 and the above-describedweight k′ have a relationship in which one is small when the other islarge.

The quantizing unit 37 quantizes the portion k×Q(x, y) of thequantization error Q(x, y) supplied from the calculating unit 36 intoN−M (=8−6=2) bits. Furthermore, the quantizing unit 37 outputs aquantized value of N−M (=2) bits obtained thereby as compensating dataQt(x, y) for compensating for error diffusion in space directions by anFRC process performed by the FRC unit 22 (FIG. 2) in the subsequentstage of the data processing unit 21.

The compensating data Qt(x, y) of N−M (=2) bits output from thequantizing unit 37 is supplied to the calculating units 34 and 39.

The dequantizing unit 38 dequantizes the ΔΣ modulation data ID(x, y) ofM (=6) bits supplied from the quantizing unit 32 into N (=8), which isthe number of bits of the original pixel value IN(x, y), and supplies itto the calculating unit 39.

That is, the dequantizing unit 38 adds 0 to the lower N−M (=2) bits ofthe ΔΣ modulation data ID(x, y) of M (=6) bits, thereby obtaining ΔΣmodulation data of N (=8) bits (hereinafter referred to as dequantizedΔΣ modulation data), and supplies the dequantized ΔΣ modulation data tothe calculating unit 39.

The calculating unit 39 adds the dequantized ΔΣ modulation data of N(=8) bits supplied from the dequantizing unit 38 and the compensatingdata Qt(x, y) of N−M (=2) bits supplied from the quantizing unit 37,thereby generating time-integration-effect-using error diffusion dataOUT(x, y) of N (=8) bits, and supplies the data to the FRC unit 22.

Here, the dequantized ΔΣ modulation data of N (=8) bits is datagenerated by adding 0 to the lower N−M (=2) bits of the ΔΣ modulationdata ID(x, y) of M (=6) bits. Therefore, thetime-integration-effect-using error diffusion data OUT(x, y) of N (=8)bits, which is obtained by adding the dequantized ΔΣ modulation data andthe compensating data Qt(x, y) of N−M (=2) bits, is data obtained byadding the compensating data Qt(x, y) of N−M (=2) bits to the lower N−M(=2) bits of the ΔΣ modulation data ID(x, y) of M (=6) bits.

Data Handled in the Gradation Converting Unit 11

Data handled in the gradation converting unit 11 in FIG. 2 is describedwith reference to FIGS. 4A to 4C.

FIG. 4A illustrates a pixel value IN(x, y) of 8 (=N) bits of targetimage data having a frame rate of 60 Hz, supplied to (the calculatingunit 31 of) the data processing unit 21.

FIG. 4B illustrates time-integration-effect-using error diffusion dataOUT(x, y) of 8 (=N) bits having a frame rate of 60 Hz, obtained for the8-bit pixel value IN (x, y) in FIG. 4A in (the calculating unit 39 of)the data processing unit 21.

As described above with reference to FIG. 3, thetime-integration-effect-using error diffusion data OUT(x, y) of 8 (=N)bits is obtained by adding the compensating data Qt(x, y) of 2 (=N−M)bits to the lower 2 (=N−M) bits of the ΔΣ modulation data ID(x, y) of 6(=M) bits.

FIG. 4C illustrates pixel values of 6 (=M) bits of gradation-convertedimage data having a frame rate of 240 Hz, obtained for thetime-integration-effect-using error diffusion data OUT(x, y) of 8 (=N)bits supplied from (the calculating unit 39 of) the data processing unit21 in the FRC unit 22.

Assume that, in the time-integration-effect-using error diffusion dataOUT(x, y), which is a pixel value of 8 bits in total, four 1-bit valuesthat correspond to the lower (=N−M) bits are represented by (b₀, b₁, b₂,and b₃), as described above. Also, assume that a 6-bit value obtained bytruncating the lower two bits of the 8-bit time-integration-effect-usingerror diffusion data OUT(x, y) is represented by X.

Here, as illustrated in FIG. 4B, the lower two bits of the 8-bittime-integration-effect-using error diffusion data OUT(x, y) is 2-bitcompensating data Qt(x, y), and a (remaining) 6-bit value X obtained bytruncating the lower two bits is 6-bit ΔΣ modulation data ID(x, y).

In the FRC process, the 8-bit time-integration-effect-using errordiffusion data OUT(x, y) is converted into four 6-bit pixel values atthe same position in a temporally-sequential four frames (pixel valuesat the position of the target pixel in sequential four frames) X+b₀,X+b₁, X+b₂, and X+b₃, as described above.

In the 6-bit pixel values X+b_(i) (i=0, 1, 2, and 3) obtained in the FRCprocess, a 6-bit value X is the ΔΣ modulation data ID(x, y) of the upper6 bits of the time-integration-effect-using error diffusion data OUT(x,y).

As described above with reference to FIG. 3, the 6-bit ΔΣ modulationdata ID(x, y) is data in which error diffusion in space directions isperformed on only a portion for the weight k′ of the quantization errorQ(x, y) in the target image data.

The 6-bit ΔΣ modulation data ID(x, y) contributes to an improvement inperceived gradation of a gradation-converted image using a spaceintegration effect due to an effect of error diffusion in spacedirections of only a portion for the weight k′ of the quantization errorQ(x, y).

The four 6-bit pixel values X+b₀, X+b₁, X+b₂, and X+b₃ obtained in theFRC process are displayed on the display 12 (FIG. 1) at a display rateof 240 Hz, but the pixel values are perceived by human vision as a sumtotal 2²×X+b₀+b₁+b₂+b₃ of X+b₀, X+b₁, X+b₂, and X+b₃ due to a timeintegration effect.

The value 2²×X in the sum total 2²×X+b₀+b₁+b₂+b₃ is equal to (thedequantized ΔΣ modulation data obtained by dequantizing) the ΔΣmodulation data ID(x, y), the upper six bits of thetime-integration-effect-using error diffusion data OUT (x, y).

Therefore, the value 2²×X in the sum total 2²×X+b₀+b₁+b₂+b₃ obtainedfrom the time integration effect contributes to an improvement inperceived gradation of a gradation-converted image using a spaceintegration effect due to an effect of error diffusion in spacedirections of only a portion for the weight k′ of the quantization errorQ(x, y).

The value b₀+b₁+b₂+b₃ in the sum total 2²×X+b₀+b₁+b₂+b₃ obtained fromthe time integration effect corresponds to the compensating data Qt(x,y) in the lower two bits of the time-integration-effect-using errordiffusion data OUT(x, y).

The compensating data Qt(x, y) is a 2-bit quantized value of the portionk×Q(x, y) of the quantization error Q(x, y) and can be expressed asQt(x, y)=(1−k′)×Q(x, y) by using the above-described weight k′. Thus,the compensating data Qt(x, y) corresponds to a portion for a weight1−k′ of the quantization error Q(x, y).

Therefore, the value b₀+b₁+b₂+b₃ in the sum total 2²×X+b₀+b₁+b₂+b₃obtained from the time integration effect corresponds to a portion forthe weight 1−k′ of the quantization error Q(x, y). As described above,the value b₀+b₁+b₂+b₃ corresponds to a portion for the weight 1−k′ ofthe quantization error Q(x, y) and has an effect of compensating for (aneffect of) error diffusion in space directions by the value 2²×X.

As described above, the compensating data Qt(x, y) compensates for errordiffusion in space directions by the value 2²×X (ΔΣ modulation dataID(x, y)) when the four 6-bit pixel values X+b₀, X+b₁, X+b₂, and X+b₃obtained after the FRC process are perceived as the sum total2²×X+b₀+b₁+b₂+b₃ due to the time integration effect.

Therefore, the ΔΣ modulation data ID(x, y) in the upper six bits of thetime-integration-effect-using error diffusion data OUT(x, y) generatesan effect of error diffusion in space directions for a portion for theweight k′ of the quantization error Q(x, y). Furthermore, thecompensating data Qt(x, y) in the lower two bits of thetime-integration-effect-using error diffusion data OUT(x, y) generatesan effect of FRC in a time direction for a portion for the weight 1−k′of the quantization error Q(x, y), and the effect compensates for theeffect of error diffusion in space directions.

As a result, according to the time-integration-effect-using errordiffusion data OUT(x, y), the entire data generates an effect of errordiffusion in space directions for the entire quantization error Q(x, y),so that perceived gradation equivalent to that in the case of errordiffusion with only ΔΣ modulation can be realized in the target imagedata (it can be prevented that a perceived gradation level of the targetimage data becomes lower than that in the case of error diffusion withonly ΔΣ modulation).

Furthermore, according to the time-integration-effect-using errordiffusion data OUT(x, y), a portion for the weight k′ of thequantization error Q(x, y) is diffused in space directions, whereas aportion for the weight 1−k′ is distributed to the four pixel valuesX+b₀, X+b₁, X+b₂, and X+b₃ that are sequential in a time direction.

That is, the quantization error Q(x, y) as noise is diffused not only inspace directions, but is diffused (distributed) in space and timedirections. Therefore, compared to a case where the quantization errorQ(x, y) as noise is diffused only in space directions, i.e., compared toa case where error diffusion with only ΔΣ modulation is performed on thetarget image data, a perceived image quality of a gradation-convertedimage displayed on the display 12 can be improved by preventingnoticeable noise in the image.

The data processing unit 21 in FIG. 3 generates the above-describedtime-integration-effect-using error diffusion data OUT(x, y) thatgenerates an effect of the error diffusion method using a timeintegration effect from the target image data.

Data Processing Performed by the Data Processing Unit 21

With reference to FIG. 5, data processing performed by the dataprocessing unit 21, that is, a process of generatingtime-integration-effect-using error diffusion data OUT(x, y), isdescribed.

The calculating unit 31 waits for and receives a pixel value of a pixelin target image data supplied thereto, and adds an output of the filter35 while regarding the pixel having the supplied pixel value as a targetpixel in step S11.

Specifically, in step S11, the calculating unit 31 adds the pixel valueof the target pixel and a value obtained through the preceding filteringperformed by the filter 35 in step S18 described below (an output of thefilter 35), and outputs a sum value obtained thereby to the quantizingunit 32 and the calculating unit 33. Then, the process proceeds to stepS12.

In step S12, the quantizing unit 32 quantizes the sum value as theoutput of the calculating unit 31, and outputs a quantized valueincluding a quantization error, the quantized value serving as ΔΣmodulation data, to the calculating unit 33 and the dequantizing unit38. Then, the process proceeds to step S13.

In step S13, the dequantizing unit 38 dequantizes the ΔΣ modulation datasupplied from the quantizing unit 32 and supplies dequantized ΔΣmodulation data to the calculating unit 39. Then, the process proceedsfrom step S13 to step S14.

In step S14, the calculating unit 33 calculates a difference between thesum value as the output of the calculating unit 31 and the output of thequantizing unit 32 (the quantized value of the sum value as the outputof the calculating unit 31, i.e., ΔΣ modulation data), thereby obtaininga quantization error of the quantization performed by the quantizingunit 32. Furthermore, the calculating unit 33 supplies the quantizationerror to the calculating units 34 and 36, and the process proceeds fromstep S14 to step S15.

In step S15, a portion of the quantization error is quantized, wherebycompensating data is generated.

Specifically, in step S15, the calculating unit 36 multiplies thequantization error supplied from the calculating unit 33 by a weight k,thereby extracting a portion of the quantization error, and supplies theextracted portion to the quantizing unit 37. The quantizing unit 37quantizes the portion of the quantization error supplied from thecalculating unit 36, thereby generating compensating data as a portionfor the weight k′ of the quantization error, and supplies thecompensating data to the calculating units 34 and 39. Then, the processproceeds from step S15 to step S16.

In step S16, the calculating unit 39 adds the dequantized ΔΣ modulationdata supplied from the dequantizing unit 38 and the compensating datasupplied from the quantizing unit 37, thereby generatingtime-integration-effect-using error diffusion data, and supplies thegenerated data to the FRC unit 22. Then, the process proceeds to stepS17.

In step S17, the calculating unit 34 calculates a difference (1−k′ ofthe quantization error) between the quantization error supplied from thecalculating unit 33 and the compensating data supplied from thequantizing unit 37, the difference serving as a ΔΣ modulation error,which is a quantization error used for ΔΣ modulation performed by the ΔΣmodulator. Then, the calculating unit 34 supplies the ΔΣ modulationerror to the filter 35, and the process proceeds to step S18.

In step S18, the filter 35 performs space-direction filtering on the ΔΣmodulation error supplied from the calculating unit 34, and supplies(outputs) a filtering result to the calculating unit 31.

Then, when a pixel value of a pixel next to the target pixel in theraster scanning order is supplied to the calculating unit 31, theprocess returns from step S18 to step S11.

In step S11, the calculating unit 31 regards the pixel next to thetarget pixel as a new target pixel, and adds the pixel value of the newtarget pixel and the filtering result supplied from the filter 35 in thepreceding step S18. Thereafter, the same process is repeated.

As described above, time-integration-effect-using error diffusion dataincluding ΔΣ modulation data (dequantized ΔΣ modulation data) andcompensating data is generated by the data processing unit 21.Accordingly, a perceived image quality of a gradation-converted imagecan be improved by preventing noticeable noise without causing adecrease in perceived gradation level of the gradation-converted image.

That is, in a case where the time-integration-effect-using errordiffusion data is displayed after the FRC process has been performedthereon, the ΔΣ modulation data ID(x, y) included in thetime-integration-effect-using error diffusion data generates an effectof error diffusion in space directions of a portion for the weight k′ ofthe quantization error. Furthermore, the compensating data included inthe time-integration-effect-using error diffusion data OUT(x, y)generates an effect of FRC in a time direction of a portion for theweight 1−k′ of the quantization error. This effect compensates for theeffect of error diffusion in space directions.

Therefore, according to the time-integration-effect-using errordiffusion data, the entire data generates an effect of error diffusionin space directions for the entire quantization error, so that perceivedgradation equivalent to that in the case of error diffusion with only ΔΣmodulation can be realized in the target image data.

Furthermore, according to the time-integration-effect-using errordiffusion data, a portion for the weight k′ of the quantization error isdiffused in space directions, whereas a portion for the weight 1−k′ isdistributed in a time direction. Therefore, a perceived image qualitycan be improved by preventing a quantization error from being noticeableas noise, compared to a case where the quantization error is diffusedonly in space directions.

In the calculating unit 36 of the data processing unit 21 (FIG. 3), theweight k multiplied by a quantization error (also weights k′ and 1−k′)can be a fixed value of 0.5, for example, or can be a variable valuethat varies in accordance with a user operation.

When the weight k is a variable value, the weight k can be set on thebasis of an analysis result obtained by analyzing target image data inunits of frames.

That is, in the data processing unit 21, a motion in a target frame (theframe of a target pixel) of target image data is detected by analyzingthe target frame, and the weight k can be set on the basis of motioninformation indicating the motion.

As the motion information, a sum of absolute differences of pixel valuesof pixels at the same position in the target frame and the precedingframe can be adopted, for example.

As the weight k, a smaller value can be set as the value of the motioninformation is larger, that is, as the motion in the target frame islarger.

In the data processing unit 21, when the weight k is large, aquantization error diffused in a time direction is large whereas aquantization error diffused in space directions is small. On the otherhand, when the weight k is small, a quantization error diffused in spacedirections is large whereas a quantization error diffused in a timedirection is small.

If most of a quantization error is diffused in a time direction when themotion in a target frame is large, a negative influence may be exertedon a gradation-converted image. For this reason, when the motion in atarget frame is large, the weight k of a small value is set so that thequantization error diffused in a time direction becomes small, asdescribed above. Accordingly, a negative influence on agradation-converted image can be prevented.

Specific Examples of the Filter 35

As the filter 35 (FIG. 3) of the data processing unit 21, a noiseshaping filter used in the error diffusion method according to a relatedart can be adopted.

Examples of the noise shaping filter used in the error diffusion methodaccording to the related art include a Jarvis, Judice & Ninke filter(hereinafter referred to as Jarvis filter) and a Floyd & Steinbergfilter (hereinafter referred to as Floyd filter).

FIG. 6 illustrates an amplitude characteristic of noise shaping usingthe Jarvis filter and an amplitude characteristic of noise shaping usingthe Floyd filter.

In FIG. 6, a contrast sensitivity curve indicating a spatial frequencycharacteristic of human vision (hereinafter also referred to as visualcharacteristic) is illustrated in addition to the amplitudecharacteristics of noise shaping.

In FIG. 6 (also in FIGS. 7, 8, 10B, 11B, and 12B described below), thehorizontal axis indicates the spatial frequency, whereas the verticalaxis indicates the gain for the amplitude characteristic or thesensitivity for the visual characteristic.

Here, the unit of the spatial frequency is cpd (cycles/degree), whichindicates the number of stripes that are seen in the range of a unitangle of view (one degree in the angle of view). For example, 10 cpdmeans that ten pairs of a white line and a black line are seen in therange of one degree in the angle of view, and 20 cpd means that twentypairs of a white line and a black line are seen in the range of onedegree in the angle of view.

A gradation-converted image generated by the gradation converting unit11 is eventually displayed on the display 12 (FIG. 1). Thus, from theviewpoint of improving the quality of the image to be displayed on thedisplay 12, it is sufficient to consider up to a maximum spatialfrequency of the image displayed on the display 12 (from 0 cpd) for thespatial frequency characteristic of human vision.

If the maximum spatial frequency of the image displayed on the display12 is very high, e.g., about 120 cpd, noise (quantization error) issufficiently modulated to a high range of a frequency band where thesensitivity of human vision is low by either of the Jarvis filter andthe Floyd filter, as illustrated in FIG. 6.

The maximum spatial frequency of the image displayed on the display 12depends on the resolution of the display 12 and the distance between thedisplay 12 and a viewer who views the image displayed on the display 12(hereinafter referred to as viewing distance).

Here, assume that the length in the vertical direction of the display 12is H inches. In this case, about 2.5 H to 3.0 H is adopted as theviewing distance to obtain the maximum spatial frequency of the imagedisplayed on the display 12.

In this case, for example, when the display 12 has a 40-inch displayscreen, having 1920 horizontal×1080 vertical pixels, for displaying aso-called full HD (High Definition) image, the maximum spatial frequencyof the image displayed on the display 12 is about 30 cpd.

FIG. 7 illustrates an amplitude characteristic of noise shaping usingthe Jarvis filter and an amplitude characteristic of noise shaping usingthe Floyd filter in a case where the maximum spatial frequency of theimage displayed on the display 12 (FIG. 1) is about 30 cpd.

FIG. 7 also illustrates a visual characteristic, as in FIG. 6.

As illustrated in FIG. 7, in the case where the maximum spatialfrequency of the image displayed on the display 12 is about 30 cpd, itis difficult for the Jarvis filter and the Floyd filter to sufficientlymodulate noise to a high range of the frequency band where thesensitivity of human vision is sufficiently low.

Therefore, when the Jarvis filter or the Floyd filter is used, noise maybe noticeable in a gradation-converted image, so that the perceivedimage quality thereof may be degraded.

In order to suppress degradation of the perceived image quality due tonoticeable noise in the gradation-converted image, the amplitudecharacteristic of noise shaping illustrated in FIG. 8 is necessary.

That is, FIG. 8 illustrates an example of an amplitude characteristic ofnoise shaping for suppressing degradation of a perceived image quality(hereinafter referred to as degradation suppressing noise shaping) dueto noticeable noise in the gradation-converted image.

Here, a noise shaping filter used for ΔΣ modulation to realize thedegradation suppressing noise shaping is also called an SBM (Super BitMapping) filter.

FIG. 8 illustrates the visual characteristic, the amplitudecharacteristic of noise shaping using the Jarvis filter, and theamplitude characteristic of noise shaping using the Floyd filterillustrated in FIG. 7, in addition to the amplitude characteristic ofthe degradation suppressing noise shaping (noise shaping using the SBMfilter).

In the amplitude characteristic of the degradation suppressing noiseshaping, the characteristic curve in a midrange and higher has anupside-down shape (including a similar shape) of the visualcharacteristic curve (contrast sensitivity curve). Hereinafter, such acharacteristic is called a reverse characteristic.

Furthermore, in the amplitude characteristic of the degradationsuppressing noise shaping, the gain increases in a high range moresteeply compared to that in the amplitude characteristic of noiseshaping using the Jarvis filter or the Floyd filter.

Accordingly, in the degradation suppressing noise shaping, noise(quantization error) is modulated to a higher range where visualsensitivity is lower in a concentrated manner, compared to the noiseshaping using the Jarvis filter or the Floyd filter.

By adopting the SBM filter as the filter 35 (FIG. 3), that is, bysetting filter coefficients of the filter 35 so that the amplitudecharacteristic of noise shaping using the filter 35 has a reversecharacteristic of the visual characteristic in the midrange and higherand that the gain increases in the high range more steeply compared tothat in the amplitude characteristic of noise shaping based on ΔΣmodulation using the Floyd filter or the Jarvis filter, noise(quantization error) in the high range where the visual sensitivity islow is added to the pixel value IN(x, y) in the calculating unit 31(FIG. 3). As a result, noise (quantization error) in thegradation-converted image can be prevented from being noticeable.

In the amplitude characteristic of noise shaping using the SBM filterillustrated in FIG. 8, the gain is well over 1 in the high range. Thismeans that the quantization error is amplified more significantly in thehigh range compared to the case where the Jarvis filter or the Floydfilter is used.

Also, in the amplitude characteristic of noise shaping using the SBMfilter illustrated in FIG. 8, the gain is negative in a low range to themidrange. Accordingly, the SBM filter can be constituted by atwo-dimensional filter having a small number of taps.

That is, in a case of realizing an amplitude characteristic in which thegain is 0 in the low range and midrange and the gain steeply increasesonly in the high range as the amplitude characteristic of noise shapingusing the SBM filter, the SBM filter is a two-dimensional filter havingmany taps (the number of taps is large).

On the other hand, in a case of realizing an amplitude characteristic ofnoise shaping using the SBM filter in which the gain is negative in thelow range or midrange, the SBM filter can be constituted by atwo-dimensional filter having a small number of taps, and the gain inthe high range of noise shaping can be increased more steeply comparedto the case of using the Jarvis filter or the Floyd filter.

Adopting such an SBM filter as the filter 35 enables the data processingunit 21 to be miniaturized.

FIG. 9 illustrates a quantization error used for filtering performed bythe SBM filter described above.

In a case where the gain is to be negative in the low range or midrangein the amplitude characteristic of noise shaping, the SBM filter can beconstituted by a 12-tap two-dimensional filter that performs filteringby using quantization errors of twelve pixels on which gradationconversion has already been performed in the raster scanning order (thepixels with downward-sloping lines in FIG. 9) among 5×5 pixels with atarget pixel being at the center.

In a case where such an SBM filter is adopted as the filter 35 (FIG. 3),the quantization error of the target pixel is diffused to pixels onwhich gradation conversion is to be performed in the raster scanningorder (prospective target pixels, i.e., the pixels with upward-slopinglines in FIG. 9) among the 5×5 pixels with the target pixel being at thecenter.

Specific Examples of Filter Coefficients and Noise ShapingCharacteristic

FIGS. 10A and 10B illustrate a first example of filter coefficients andan amplitude characteristic of noise shaping using the SBM filter in acase where the maximum spatial frequency of the image displayed on thedisplay 12 that displays a gradation-converted image is 30 cpd.

Specifically, FIG. 10A illustrates a first example of filtercoefficients of the 12-tap SBM filter, the filter coefficients beingdetermined so that the gain in the amplitude characteristic of noiseshaping is negative in the low range or midrange and increases in thehigh range more steeply compared to that in the amplitude characteristicof noise shaping based on ΔΣ modulation using the Floyd filter.

In FIG. 10A, filter coefficients g(1, 1)=−0.0317, g(2, 1)=−0.1267, g(3,1)=−0.1900, g(4, 1)=−0.1267, g(5, 1)=−0.0317, g(1, 2)=−0.1267, g(2,2)=0.2406, g(3, 2)=0.7345, g(4, 2)=0.2406, g(5, 2)=−0.1267, g(1,3)=−0.1900, and g(2, 3)=0.7345 are adopted as the filter coefficients ofthe 12-tap SBM filter.

Here, the SBM filter is a two-dimensional FIR filter. The filtercoefficient g(i, j) is a filter coefficient multiplied by a quantizationerror of the pixel i-th from the left and j-th from the top in 12 pixelson which gradation conversion has already been performed in the rasterscanning order among 5 horizontal×5 vertical pixels with a target pixelbeing at the center described above with reference to FIG. 9.

FIG. 10B illustrates an amplitude characteristic of noise shaping usingthe SBM filter in a case where the SBM filter has the filtercoefficients illustrated in FIG. 10A.

In the amplitude characteristic of noise shaping in FIG. 10B, the gainis 0 when the frequency f is 0, the gain is negative in the low range ormidrange, and the gain increases in the high range more steeply comparedto that in the amplitude characteristic of noise shaping based on ΔΣmodulation using the Floyd filter (and the Jarvis filter).

FIGS. 11A and 11B illustrate a second example of filter coefficients andan amplitude characteristic of noise shaping using the SBM filter in acase where the maximum spatial frequency of the image displayed on thedisplay 12 that displays a gradation-converted image is 30 cpd.

Specifically, FIG. 11A illustrates a second example of filtercoefficients of the 12-tap SBM filter, the filter coefficients beingdetermined so that the gain in the amplitude characteristic of noiseshaping is negative in the low range or midrange and increases in thehigh range more steeply compared to that in the amplitude characteristicof noise shaping based on ΔΣ modulation using the Floyd filter.

In FIG. 11A, filter coefficients g(1, 1)=−0.0249, g (2, 1)=−0.0996, g(3,1)=−0.1494, g(4, 1)=−0.0996, g(5, 1)=−0.0249, g(1, 2)=−0.0996, g(2,2)=0.2248, g(3, 2)=0.6487, g(4, 2)=0.2248, g(5, 2)=−0.0996, g(1,3)=−0.1494, and g(2, 3)=0.6487 are adopted as the filter coefficients ofthe 12-tap SBM filter.

FIG. 11B illustrates an amplitude characteristic of noise shaping usingthe SBM filter in a case where the SBM filter has the filtercoefficients illustrated in FIG. 11A.

In the amplitude characteristic of noise shaping in FIG. 11B, the gainis 0 when the frequency f is 0, the gain is negative in the low range ormidrange, and the gain increases in the high range more steeply comparedto that in the amplitude characteristic of noise shaping based on ΔΣmodulation using the Floyd filter.

FIGS. 12A and 12B illustrate a third example of filter coefficients andan amplitude characteristic of noise shaping using the SBM filter in acase where the maximum spatial frequency of the image displayed on thedisplay 12 that displays a gradation-converted image is 30 cpd.

Specifically, FIG. 12A illustrates a third example of filtercoefficients of the 12-tap SBM filter, the filter coefficients beingdetermined so that the gain in the amplitude characteristic of noiseshaping is negative in the low range or midrange and increases in thehigh range more steeply compared to that in the amplitude characteristicof noise shaping based on ΔΣ modulation using the Floyd filter.

In FIG. 12A, filter coefficients g(1, 1)=−0.0397, g (2, 1)=−0.1586, g(3,1)=−0.2379, g(4, 1)=−0.1586, g(5, 1)=−0.0397, g(1, 2)=−0.1586, g(2,2)=0.2592, g(3, 2)=0.8356, g(4, 2)=0.2592, g(5, 2)=−0.1586, g(1,3)=−0.2379, and g(2, 3)=0.8356 are adopted as the filter coefficients ofthe 12-tap SBM filter.

FIG. 12B illustrates an amplitude characteristic of noise shaping usingthe SBM filter in a case where the SBM filter has the filtercoefficients illustrated in FIG. 12A.

In the amplitude characteristic of noise shaping in FIG. 12B, the gainis 0 when the frequency f is 0, the gain is negative in the low range ormidrange, and the gain increases in the high range more steeply comparedto that in the amplitude characteristic of noise shaping based on ΔΣmodulation using the Floyd filter.

The filter coefficients of the 12-tap SBM filter illustrated in FIGS.10A, 11A, and 12A include negative values, and thus the gain in theamplitude characteristic of noise shaping is negative in the low rangeor midrange. In this way, by allowing the gain in the amplitudecharacteristic of noise shaping to be negative in the low range ormidrange, the amplitude characteristic of noise shaping in which thegain steeply increases in the high range can be realized by an SBMfilter having a small number of taps, such as 12 taps.

Additionally, according to a simulation that was performed by using SBMfilters having the filter coefficients illustrated in FIGS. 10A, 11A,and 12A as the filter 35, a gradation-converted image having a highperceived quality could be obtained in all of the SBM filters.

Descriptions have been given about a case where an embodiment of thepresent invention is applied to the image processing apparatus (FIG. 1)that performs gradation conversion on an 8-bit target image to generatea 6-bit image and that displays the 6-bit image on the display 12, whichis a 6-bit LCD. However, the embodiment of the present invention canalso be applied to other cases of performing gradation conversion on animage.

For example, in a case of performing color space conversion ofconverting an image in which each of YUV components is 8 bits into animage having each of RGB components as a pixel value and then displayingthe image that has been obtained through the color space conversion andthat has RGB components as a pixel value on an 8-bit LCD, an image inwhich each of RGB components exceeds the original 8 bits, e.g., expandedto 16 bits, may be obtained through the color space conversion. In thiscase, it is necessary to perform gradation conversion on the image inwhich each of RGB components has been expanded to 16 bits in order toobtain an 8-bit image that can be displayed on the 8-bit LCD. Theembodiment of the present invention can also be applied to suchgradation conversion.

Exemplary Configuration of a Computer According to an Embodiment of thePresent Invention

The above-described series of processes can be performed by either ofhardware and software. When the series of processes are performed bysoftware, a program constituting the software is installed to amulti-purpose computer or the like.

FIG. 13 illustrates an exemplary configuration of a computer to whichthe program for executing the above-described series of processes isinstalled according to an embodiment.

The program can be recorded in advance in a hard disk 105 or a ROM (ReadOnly Memory) 103 serving as a recording medium mounted in the computer.

Alternatively, the program can be stored (recorded) temporarily orpermanently in a removable recording medium 111, such as a flexibledisk, a CD-ROM (Compact Disc Read Only Memory), an MO (Magneto Optical)disc, a DVD (Digital Versatile Disc), a magnetic disk, or asemiconductor memory. The removable recording medium 111 can be providedas so-called package software.

The program can be installed to the computer via the above-describedremovable recording medium 111. Also, the program can be transferred tothe computer from a download site via an artificial satellite fordigital satellite broadcast in a wireless manner, or can be transferredto the computer via a network such as a LAN (Local Area Network) or theInternet in a wired manner. The computer can receive the programtransferred in that manner by using a communication unit 108 and caninstall the program to the hard disk 105 mounted therein.

The computer includes a CPU (Central Processing Unit) 102. Aninput/output interface 110 is connected to the CPU 102 via a bus 101.When a command is input to the CPU 102 by a user operation of an inputunit 107 including a keyboard, a mouse, and a microphone via theinput/output interface 110, the CPU 102 executes the program stored inthe ROM 103 in response to the command. Alternatively, the CPU 102loads, to a RAM (Random Access Memory) 104, the program stored in thehard disk 105, the program transferred via a satellite or a network,received by the communication unit 108, and installed to the hard disk105, or the program read from the removable recording medium 111 loadedinto a drive 109 and installed to the hard disk 105, and executes theprogram. Accordingly, the CPU 102 performs the process in accordancewith the above-described flowchart or the process performed by theabove-described configurations illustrated in the block diagrams. Then,the CPU 102 allows an output unit 106 including an LCD (Liquid CrystalDisplay) and a speaker to output, allows the communication unit 108 totransmit, or allows the hard disk 105 to record a processing result viathe input/output interface 110 as necessary.

In this specification, the process steps describing the program allowingthe computer to execute various processes are not necessarily performedin time series along the order described in a flowchart, but may beperformed in parallel or individually (e.g., a parallel process or aprocess by an object is acceptable).

The program may be processed by a single computer or may be processed ina distributed manner by a plurality of computers. Furthermore, theprogram may be executed by being transferred to a remote computer.

Embodiments of the present invention are not limited to theabove-described embodiments, and various modifications are acceptablewithout deviating from the scope of the present invention.

For example, in the above-described embodiment, the gradation convertingunit 11 performs gradation conversion on 8-bit target image data havinga frame rate of 60 Hz to eventually obtain 6-bit gradation-convertedimage data having a frame rate of 240 Hz, four times 60 Hz. However,when the target image data is substantially still image data (e.g., dataof an image with no object moving at high speed), the gradationconverting unit 11 can convert the 8-bit target image data having aframe rate of 60 Hz into 6-bit gradation-converted image data having aframe rate of 60 Hz.

For example, in a case where the image processing apparatus illustratedin FIG. 1 is applied to a so-called notebook PC (Personal Computer) inwhich an image of an object moving at high speed is less likely to bedisplayed, the data processing unit 21 of the gradation converting unit11 performs data processing on every fourth frame of the 8-bit targetimage data having a frame rate of 60 Hz, thereby generatingtime-integration-effect-using error diffusion data having a frame rateof quarter of 60 Hz. Then, an FRC process is performed on thetime-integration-effect-using error diffusion data having a frame rateof quarter of 60 Hz, whereby 6-bit gradation-converted image data havingan original frame rate of 60 Hz can be obtained.

In this case, the FRC unit 22 that handles image data of a high framerate, such as 240 Hz, is unnecessary. Furthermore, an LCD having adisplay rate of 60 Hz, not an LCD having a high display rate of 240 Hz,can be adopted as the display 12.

It should be understood by those skilled in the art that variousmodifications, combinations, sub-combinations and alterations may occurdepending on design requirements and other factors insofar as they arewithin the scope of the appended claims or the equivalents thereof.

What is claimed is:
 1. An image processing apparatus comprising: firstcalculating means for adding a pixel value of an image and an output offilter means for performing filtering in space directions on amodulation error of a quantized value obtained by quantizing the pixelvalue of the image; first quantizing means for quantizing an output ofthe first calculating means and outputting a quantized value serving asΔΣ modulation data, which is a result of ΔΣ modulation performed on thepixel value; second calculating means for calculating a differencebetween the output of the first calculating means and the output of thefirst quantizing means, thereby obtaining a quantization error; secondquantizing means for quantizing a portion of the quantization error andoutputting a quantized value obtained through the quantization, thequantized value serving as compensating data for compensating for errordiffusion in space directions; dequantizing means for dequantizing theΔΣ modulation data; third calculating means for adding the dequantizedΔΣ modulation data and the compensating data, thereby generatingtime-integration-effect-using error diffusion data that generates aneffect of an error diffusion method using a visual integration effect ina time direction; fourth calculating means for calculating a differencebetween the quantization error and the compensating data, the differenceserving as a ΔΣ modulation error, which is a quantization error used forthe ΔΣ modulation; and the filter means for performing filtering inspace directions on the ΔΣ modulation error.
 2. The image processingapparatus according to claim 1, wherein, in a case where the pixel valueof the image is N bits and where the first quantizing means outputs aquantized value of M bits (M smaller than N) the ΔΣ modulation data, thedequantizing means dequantizes the ΔΣ modulation data into N bits; thesecond quantizing means quantizes the portion of the quantization errorinto N-M bits and outputs a quantized portion obtained thereby asN-M-bit compensating data, and the third calculating means adds the ΔΣmodulation data the third calculating means adds the N-bit dequantizedΔΣ modulation data and the N-M-bit compensating data, thereby generatingN-bit time-integration-effect-using error diffusion data.
 3. The imageprocessing apparatus according to claim 2, further comprising: framerate control means for performing a converting process of converting thetime-integration-effect-using error diffusion data of N bits into apixel value of M bits by using a frame rate control process.
 4. Theimage processing apparatus according to claim 1, wherein filtercoefficients of filtering performed by the filter means are determinedso that an amplitude characteristic of noise shaping performed based onthe ΔΣ modulation becomes a reverse characteristic of a human visualcharacteristic in a midrange and higher and that a gain in the amplitudecharacteristic increases in a high range more steeply compared to a gainin an amplitude characteristic of noise shaping performed based on ΔΣmodulation using a Floyd filter.
 5. The image processing apparatusaccording to claim 1, wherein filter coefficients of filtering performedby the filter means are determined so that a gain in an amplitudecharacteristic of noise shaping performed based on the ΔΣ modulation isnegative in a low range or midrange and increases in a high range moresteeply compared to a gain in an amplitude characteristic of noiseshaping performed based on ΔΣ modulation using a Floyd filter.
 6. Theimage processing apparatus according to claim 1, wherein filtercoefficients of filtering performed by the filter means include anegative value and are determined so that a gain in an amplitudecharacteristic of noise shaping performed based on the ΔΣ modulationincreases in a high range more steeply compared to a gain in anamplitude characteristic of noise shaping performed based on ΔΣmodulation using a Floyd filter.
 7. An image processing method for animage process apparatus including first calculating means for adding apixel value of an image and an output of filter means for performingfiltering in space directions on a modulation error of a quantized valueobtained by quantizing the pixel value of the image; first quantizingmeans for quantizing an output of the first calculating means andoutputting a quantized value serving as ΔΣ modulation data, which is aresult of ΔΣ modulation performed on the pixel value; second calculatingmeans for calculating a difference between the output of the firstcalculating means and the output of the first quantizing means, therebyobtaining a quantization error; second quantizing means for quantizing aportion of the quantization error and outputting a quantized valueobtained through the quantization, the quantized value serving ascompensating data for compensating for error diffusion in spacedirections; dequantizing means for dequantizing the ΔΣ modulation data;third calculating means for adding the dequantized ΔΣ modulation dataand the compensating data, thereby generatingtime-integration-effect-using error diffusion data that generates aneffect of an error diffusion method using a visual integration effect ina time direction; fourth calculating means for calculating a differencebetween the quantization error and the compensating data, the differenceserving as a ΔΣ modulation error, which is a quantization error used forthe ΔΣ modulation; and the filter means for performing filtering inspace directions on the ΔΣ modulation error, the image processing methodcomprising the steps of: adding the pixel value of the image and theoutput of the filter means, the adding being performed by the firstcalculating means; quantizing the output of the first calculating meansand outputting the quantized value including the quantization error, thequantized value serving as the ΔΣ modulation data, the quantizing andthe outputting being performed by the first quantizing means;calculating the difference between the output of the first calculatingmeans and the quantized value of the output of the first calculatingmeans, thereby obtaining the quantization error, the calculating beingperformed by the second calculating means; quantizing the portion of thequantization error and outputting the compensating data, the quantizingand the outputting being performed by the second quantizing means;dequantizing the ΔΣ modulation data; adding the dequantized ΔΣmodulation data and the compensating data, thereby generating thetime-integration-effect-using error diffusion data, the adding beingperformed by the third calculating means; calculating the differencebetween the quantization error and the compensating data, the differenceserving as the ΔΣ modulation error, the calculating being performed bythe fourth calculating means; and performing filtering in spacedirections on the ΔΣ modulation error, the performing being performed bythe filter means.
 8. A non-transitory computer-readable medium havingstored thereon a computer-readable program causing a computer to carryout the following steps first calculating step for adding a pixel valueof an image and an output of filter step for performing filtering inspace directions on a modulation error of a quantized value obtained byquantizing the pixel value of the image; first quantizing step forquantizing an output of the first calculating step and outputting aquantized value serving as ΔΣ modulation data, which is a result of ΔΣmodulation performed on the pixel value; second calculating step forcalculating a difference between the output of the first calculatingstep and the output of the first quantizing step, thereby obtaining aquantization error; second quantizing step for quantizing a portion ofthe quantization error and outputting a quantized value obtained throughthe quantization, the quantized value serving as compensating data forcompensating for error diffusion in space directions; dequantizing meansfor dequantizing the ΔΣ modulation data; third calculating step foradding the dequantized ΔΣ modulation data and the compensating data,thereby generating time-integration-effect-using error diffusion datathat generates an effect of an error diffusion method using a visualintegration effect in a time direction; fourth calculating step forcalculating a difference between the quantization error and thecompensating data, the difference serving as a ΔΣ modulation error,which is a quantization error used for the ΔΣ modulation; and the filterstep for performing filtering in space directions on the ΔΣ modulationerror.
 9. An image processing apparatus comprising: a first calculatingunit configured to add a pixel value of an image and an output of afilter unit configured to perform filtering in space directions on amodulation error of a quantized value obtained by quantizing the pixelvalue of the image; a first quantizing unit configured to quantize anoutput of the first calculating unit and output a quantized valueserving as ΔΣ modulation data, which is a result of ΔΣ modulationperformed on the pixel value; a second calculating unit configured tocalculate a difference between the output of the first calculating unitand the output of the first quantizing unit, thereby obtaining aquantization error; a second quantizing unit configured to quantize aportion of the quantization error and output a quantized value obtainedthrough the quantization, the quantized value serving as compensatingdata for compensating for error diffusion in space directions;dequantizing means for dequantizing the ΔΣ modulation data; a thirdcalculating unit configured to add the dequantized ΔΣ modulation dataand the compensating data, thereby generatingtime-integration-effect-using error diffusion data that generates aneffect of an error diffusion method using a visual integration effect ina time direction; a fourth calculating unit configured to calculate adifference between the quantization error and the compensating data, thedifference serving as a ΔΣ modulation error, which is a quantizationerror used for the ΔΣ modulation; and the filter unit configured toperform filtering in space directions on the ΔΣ modulation error.